Label embedding for text recognition

The standard approach to recognizing text in images consists in first classifying local image regions into candidate characters and then combining them with high-level word models such as conditional random fields (CRF). This paper explores a new paradigm that departs from this bottom-up view. In our approach, every label from a lexicon is embedded to an Euclidean vector space. We refer to this step as label embedding. Each vector of image features is then projected to this space. To that end, we formulate the problem in a structured support vector machine (SSVM) framework [3] and learn the linear projection that optimizes a proximity criterion between word images and their corresponding labels: matching label-image pairs should be closer than non-matching pairs. In this space, the "compatibility" between a word image and a label is measured simply as the dot product between their representations. Therefore, given a new word image, recognition amounts to finding the closest label in the common space (Fig. 1).